Logic, self-awareness and self-improvement: The metacognitive loop and the problem of brittleness

This essay describes a general approach to building perturbation-tolerant autonomous systems, based on the conviction that artificial agents should be able notice when something is amiss, assess the anomaly, and guide a solution into place. We call this basic strategy of self-guided learning the metacognitive loop; it involves the system monitoring, reasoning about, and, when necessary, altering its own decision-making components. In this essay, we (a) argue that equipping agents with a metacognitive loop can help to overcome the brittleness problem, (b) detail the metacognitive loop and its relation to our ongoing work on time-sensitive commonsense reasoning, (c) describe specific, implemented systems whose perturbation tolerance was improved by adding a metacognitive loop, and (d) outline both short-term and long-term research agendas

The roots of self-awareness

In this paper we provide an account of the structural underpinnings of self-awareness. We offer both an abstract, logical account-by way of suggestions for how to build a genuinely self-referring artificial agent-and a biological account, via a discussion of the role of somatoception in supporting and structuring self-awareness more generally. Central to the account is a discussion of the necessary motivational properties of self-representing mental tokens, in light of which we offer a novel definition of self-representation. We also discuss the role of such tokens in organizing self-specifying information, which leads to a naturalized restatement of the guarantee that introspective awareness is immune to error due to mis-identification of the subject